Certified Convergent Perceptron Learning

Frank Rosenblatt invented the Perceptron algorithm in 1957 as part of an early attempt to build “brain models” – artificial neural networks. In this paper, we apply tools from symbolic logic – dependent type theory as implemented in the interactive theorem prover Coq – to prove that one-layer perceptrons for binary classification converge when trained on linearly separable datasets (the Perceptron convergence theorem). We perform experiments to evaluate the performance of our Coq Perceptron vs. a C++ implementation and against a hybrid implementation in which separators learned in C++ are certified in Coq. We find that by carefully optimizing the extraction of our Coq perceptron, we can meet – and occasionally exceed – the performance of the C++ implementation. Our work is both proof engineering and intellectual archaeology: Even classic machine learning algorithms (and to a lesser degree, termination proofs) are understudied in the interactive theorem proving literature. At the same time, recasting Perceptron and its convergence proof in the language of 21st century human-assisted theorem provers may illuminate, for a fresh audience, a small but interesting corner of the history of ideas.